Webcam-based Eye Gaze Tracking under Natural Head Movement

The estimation of the human’s point of visual gaze is important for many applications. This information can be used in visual gaze based human-computer interaction, advertisement, human cognitive state analysis, attentive interfaces, human behavior analysis. Visual gaze direction can also provide high-level semantic cues such as who is speaking to whom, information on non-verbal communication and the mental state/attention of a human (e.g., a driver). Overall, the visual gaze direction is important to understand the human’s attention, motivation and intention. There is a tremendous amount of research concerned with the estimation of the point of visual gaze first attempts can be traced back 1989. Many visual gaze trackers are offered to this date, although they all share the same drawbacks either they are highly intrusive (head mounted systems, electro-ocolography methods, sclera search coil methods), either they restrict the user to keep his/her head as static as possible (PCCR methods, neural network methods, ellipse fitting methods) or they rely on expensive and sophisticated hardware and prior knowledge in order to grant the freedom of natural head movement to the user (stereo vision methods, 3D eye modeling methods). Furthermore, all proposed visual gaze trackers require an user specific calibration procedure that can be uncomfortable and erroneous, leading to a negative impact on the accuracy of the tracker. Although, some of these trackers achieve extremely high accuracy, they lack of simplicity and can not be efficiently used in everyday life. This manuscript investigates and proposes a visual gaze tracker that tackles the problem using only an ordinary web camera and no prior knowledge in any sense (scene set-up, camera intrinsic and/or extrinsic parameters). The tracker we propose is based on the observation that our desire to grant the freedom of natural head movement to the user requires 3D modeling of the scene set-up. Although, using a single low resolution web camera bounds us in dimensions (no depth can be recovered), we propose ways to cope with this drawback and model the scene in front of the user. We tackle this three-dimensional problem by